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There has recently been considerable interests in design of low- 
complexity, myopic, distributed and stable scheduling policies for con- 
strained queueing network models that arise in the context of emerg- 
ing communication networks. Here, we consider two representative 
models. One, a model for the collection of wireless nodes communi- 
cating through a shared medium, that represents randomly varying 
number of packets in the queues at the nodes of networks. Two, a 
buffered circuit switched network model for an optical core of fu- 
ture Internet, to capture the randomness in calls or flows present 
in the network. The maximum weight scheduling policy proposed 
by Tassiulas and Ephremide [32] leads to a myopic and stable pol- 
icy for the packet-level wireless network model. But computationally 
it is very expensive (NP-hard) and centralized. It is not applicable 
to the buffered circuit switched network due to the requirement of 
non-premption of the calls in the service. As the main contribution 
of this paper, we present a stable scheduling algorithm for both of 
these models. The algorithm is myopic, distributed and performs few 
logical operations at each node per unit time. 

1. Introduction. The primary task of a communication network archi- 
tect is to provision as well as utilize network resources efficiently to satisfy 
the demands imposed on it. The main algorithmic problem is that of al- 
locating or scheduling resources among various entities or data units, e.g. 
packets, flows, that are contending to access them. In recent years, the ques- 
tion of designing a simple, myopic, distributed and high-performance (aka 
stable) scheduling algorithm has received considerable interest in the con- 
text of emerging communication network models. Two such models that we 
consider this paper are that of a wireless network and a buffered circuit 
switched network. 
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The wireless network consists of wireless transmission capable nodes. Each 
node receives exogenous demand in form of packets. These nodes communi- 
cate these packets through a shared wireless medium. Hence their simultane- 
ous transmission may contend with each other. The purpose of a scheduling 
algorithm is to resolve these contentions among transmitting nodes so as to 
utilize the wireless network bandwidth efficiently while keeping the queues 
at nodes finite. Naturally the desired scheduling algorithm should be dis- 
tributed, simple/low-complexity and myopic (i.e. utilize only the network 
state information like queue-sizes). 

The buffered circuit switched network can be utilized to model the dy- 
namics of flows or calls in an optical core of future Internet. Here a link 
capacitated network is given with a collection of end-to-end routes. At the 
ingress (i.e. input or entry point) of each route, calls arriving as per ex- 
ogenous process are buffered or queued. Each such call desires resources on 
each link of its route for a random amount of time duration. Due to link 
capacity constraints, calls of routes sharing links contend for resources. And, 
a scheduling algorithm is required to resolve this contention so as to utilize 
the network links efficiently while keeping buffers or queues at ingress of 
routes finite. Again, the scheduling algorithm is desired to be distributed, 
simple and myopic. 

An important scheduling algorithm is the maximum weight policy that 
was proposed by Tassiulas and Ephremides [32]. It was proposed in the 
context of a packet queueing network model with generic scheduling con- 
straints. It is primarily applicable in a scenario where scheduling decisions 
are synchronized or made every discrete time. It suggests scheduling queues, 
subject to constraints, that have the maximum net weight at each time step 
with the weight of a queue being its queue-size. They established through- 
put optimality property of this algorithm for this general class of networks. 
Further, this algorithm, as the description suggests, is myopic. Due to the 
general applicability and myopic nature, this algorithm and its variants have 
received a lot of attention in recent years, e.g. [201 EOl EZl HI [28]. 

The maximum weight algorithm provides a myopic and stable schedul- 
ing algorithm for the wireless network model. However, it requires solving 
a combinatorial optimization problem, the maximum weight independent 
set problem, to come up with a schedule every time. And the problem of 
finding a maximum weight independent set is known to be NP-hard as well 
as hard to approximate in general [33] . To address this concern, there has 
been a long line of research conducted to devise implementable approxima- 
tions of the maximum weight scheduling algorithm, e.g. [191 Ell HI El l23] . 
A comprehensive survey of such maximum weight inspired and other algo- 
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rithmic approaches that have been studied over more than four decades in 
the context of wireless networks can be found in [25\ [T5] . 

In the context of buffered circuit switched network, calls have random ser- 
vice requirement. Therefore, scheduling decisions can not be synchronized. 
Therefore, the maximum weight scheduling algorithm is not applicable. To 
the best of our knowledge, no other myopic and stable algorithm is known 
for this network model. 

1.1. Contributions. We propose a scheduling algorithm for both wireless 
and buffered circuit switched network model. The algorithm utilizes only 
local, queue-size information to make scheduling decisions. That is, the al- 
gorithm is myopic and distributed. It requires each queue (or node) in the 
network to perform few (literally, constant) logical operations per scheduling 
decision. We establish that it is throughput optimal. That is, the network 
Markov process is positive Harris recurrent as long as the network is under- 
loaded (or not overloaded). 

Philosophically, our algorithm design is motivated by a certain product- 
form distribution that can be characterized as the stationary distribution of 
a simple and distributed Markovian dynamics over the space of schedules. 
For the wireless network, it corresponds to the known Glauber dynamics (cf. 
|18j ) over the space of independent sets of the wireless network interference 
graph; for the buffered circuit switched network, it corresponds to the known 
stochastic loss network (cf. [H]). 

To establish the stability property of the algorithm, we exhibit an ap- 
propriate Lyapunov function. This, along with standard machinery based 
on identifying an appropriate 'petit set', leads to the positive Harris re- 
currence property of the network Markov process. Technically, this is the 
most challenging part of our result. It requires proving an effective 'time 
scale separation' between the network queuing dynamics and the scheduling 
dynamics induced by the algorithm. To make this possible, we use an appro- 
priately slowly increasing function (loglog(- + e)) of queue-size as weight in 
the scheduling algorithm. Subsequently, the time scale separation follows by 
studying the mixing property of a specific time varying Markov chain over 
the space of schedules. 

We note that use of Lyapunov function for establishing stability is some- 
what classical now (for example, see [321 130^ [27]). Usually difficulty lies in 
finding an appropriate candidate function followed by establishing that it is 
indeed a "Lyapunov" function. 

1.2. Organization. We start by describing two network models, the wire- 
less network and the buffered circuit switched network in Section [2j We 
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formally introduce the problem of scheduling and performance metric for 
scheduling algorithms. The maximum weight scheduling algorithm is de- 
scribed as well. Our randomized algorithm and its throughput optimality for 
both network models are presented in Section [3j The paper beyond Section 
[3] is dedicated to establishing the throughput optimality. Necessary techni- 
cal preliminaries are presented in Section [H Here we relate our algorithm 
for both models with appropriate reversible Markov chains on the space 
of schedules and state useful properties of these Markov chains. We also 
describe known facts about the positive Harris recurrence as well as state 
the known Lyapunov drift criteria, to establish positive Harris recurrence. 
Detailed proofs of our main results are presented in Section [5j 

2. Setup. 

2.1. Wireless Network. We consider a single-hop wireless network of n 
queues. Queues receive work as per exogenous arrivals and work leaves the 
system upon receiving service. Specifically, let Qi{t) £ M+ = {x G M : x > 0} 
denote the amount of work in the iih queue at time t G M+ and Q(t) = 
[Qi{t)]i<i<n', initially t = and Q(0) = C0. Work arrives to each queue in 
terms of unit-sized packets as per a discrete-time process. Let Ai{s, t) denote 
the amount of work arriving to queue i in time interval [s, t] for < s < t. For 
simplicity, assume that for each i, Ai{-) is an independent Bernoulli process 

with parameter Xi, where Ai{T) = Aj(0, r). That is, Ai{T+l)—Ai{T) £ {0,1} 
and Pr(Ai(T + 1) - Aj(r) = 1) = for ah i and t e Z+ = {k e Z : k > 0}. 
Denote the arrival rate vector as A = [Aj]i<j<„. We assume that arrivals 
happen at the end of a time slot. 

The work from queues is served at the unit rate, but subject to inter- 
ference constraints. Specifically, let G = (V, E) denote the inference graph 
between the n queues, represented by vertices F = {1, . . . n} and edges E: 
an G E implies that queues i and j can not transmit simultaneously 
since their transmission interfere with each other. Formally, let ai{t) G {0, 1} 
denotes whether the queue i is transmitting at time t, i.e. work in queue i 
is being served at unit rate at time t and cr(t) = [(Ti(t)]. Then, it must be 
that for t G M+, 

a{t) G 1{G) = {p= [pi] G {0, 1}" ■.p^ + PJ<l for all (i, j) G E}. 
The total amount of work served at queue i in time interval [s,t] is 



Di{s,t) = 




^Bold letters are reserved for vectors; 0, 1 represent vectors of all Os & all Is respectively. 
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where I^^} denotes the indicator function. 

In summary, the above model induces the following queueing dynamics: 
for any < s < t and 1 < i < n, 

Qi{t) = Qi{s) - I ai{y)ls^Q^^y)^Q}dy + Ai{s,t). 

J s 

2.2. Buffered Circuit Switched Network. We consider a buffered circuit 
switched network. Here the network is represented by a capacitated graph 
G = iy, E) with V being vertices, E <Z V ^ V being links (or edges) with 
each link e £ E having a finite integral capacity Ce G N. This network is 
accessed by a fixed set of n routes each route is a collection 
of interconnected links. At each route Ri, flows arrive as per an exogenous 
arrival process. For simplicity, we assume it to be an independent Possion 
process of rate Aj and let Ai{s,t) denote total number of flow arrivals to 
route Ri in time interval [s, t]. Upon arrival of a fiow to route Ri, it joins the 
queue or buffer at the ingress of Ri. Let Qi{t) denote the number of fiows 
in this queue at time t; initially t = and Qi{0) = 0. 

Each flow arriving to Ri, comes with the service requirement of using 
unit capacity simultaneously on all the links of Ri for a time duration - it is 
assumed to be distributed independently as per Exponential of unit mean. 
Now a flow in the queue of route Ri can get simultaneous possession of links 
along route Ri in the network at time t, if there is a unit capacity available 
at all of these links. To this end, let Zi{t) denote the number of flows that are 
active along route Ri, i.e. posses links along the route Ri. Then, by capacity 
constraints on the links of the network, it must be that z{t) = [zi{t)] satisfies 

z{t) eX = {z = [z,i] ^ Zi<Ce, V e G E}. 

This represents the scheduling constraints of the circuit switched network 
model similar to the interference constraints of the wireless network model. 

Finally, a fiow active on route Ri , departs the network after the completion 
of its service requirement and frees unit capacity on the links of Ri. Let 
Di{s, t) denote the number of fiows which are served (hence leave the system) 
in time interval [s,t\. 

2.3. Scheduling Algorithm & Performance Metric. In both models de- 
scribed above, the scheduling is the key operational question. In the wireless 
network, queues need to decide which of them transmit subject to interfer- 
ence constraints. In the circuit switched network, queues need to agree on 
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which flows becomes active subject to network capacity constraints. And, a 
scheduling algorithm is required to make these decisions every time. 

In wireless network, the scheduling algorithm decides the schedule cr(t) € 
at each time t. We are interested in distributed scheduling algorithms, 
i.e. queue i decides cTj(t) using its local information, such as its queue-size 
Qi{t). We assume that queues have instantaneous carrier sensing informa- 
tion, i.e. if a queue (or node) j starts transmitting at time t, then all neigh- 
boring queues can listen to this transmission immediately. 

In buffered circuit switched network, the scheduling algorithm decides ac- 
tive flows or schedules z(t) at time t. Again, our interest is in distributed 
scheduling algorithms, i.e. queue at ingress of route Ri decides Zi{t) using 
its local information. Each queue (or route) can obtain instantaneous infor- 
mation on whether all links along its route have unit capacity available or 
not. 

In summary, both models need scheduling algorithms to decide when each 
queue (or its ingress port) will request the network for availability of re- 
sources; upon a positive answer (or successful request) from the network, 
the queue acquires network resources for certain amount of time. And, these 
algorithm need to be based on local information. 

From the perspective of network performance, we would like the schedul- 
ing algorithm to be such that the queues in network remain as small as 
possible for the largest possible range of arrival rate vectors. To formalize 
this notion of performance, we define the capacity regions for both of these 
models. Let A^„ be the capacity region of the wireless network model defined 
as 

A^ = Conv(X(G)) 

(1) = < y G IR+ : y < ^ Oia-cr, with aa- > 0, and ^ a^- < I 

<tGX(G) (tGX(G) 

And let Acs be the capacity region of the buffered circuit switched network 
defined as 



A. 

(2) 




azZ, with az > 0, and ciz < 1 > . 



Intuitively, these bounds of capacity regions comes from the fact that any 
algorithm produces the 'service rate' from X[G) (or X) each time and hence 
the time average of the service rate induced by any algorithm must belong 
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to its convex hull. Therefore, if arrival rates A can be 'served well' by any 
algorithm then it must belong to Conv(X(G)) (or Conv(A')). 

Motivated by this, we call an arrival rate vector A admissible if A G A, 
and say that an arrival rate vector A is strictly admissible if A G A°, where 
A" is the interior of A formally defined as 

A° = {a G M" : A < A* componentwise, for some A* G A} . 

Equivalently, we may say that the network is under-loaded. Now we are ready 
to define a performance metric for a scheduling algorithm. Specifically, we 
desire the scheduling algorithm to be throughput optimal as defined below. 

Definition 1 (throughput optimal) A scheduling algorithm is called 
throughput optimal, or stable, or providing 100% throughput, if for any 
A G A° the (appropriately defined) underlying network Markov process is 
positive (Harris) recurrent. 

2.4. The MW Algorithm. Here we describe a popular algorithm known 
as the maximum weight or in short MW algorithm that was proposed by 
Tassiulas and Ephremides [32]. It is throughput optimal for a large class 
of network models. The algorithm readily applies to the wireless network 
model. However, it does not apply (exactly or any variant of it) in the case 
of circuit switched network. Further, this algorithm requires solving a hard 
combinatorial problem each time slot, e.g. maximum weight independent set 
for wireless network, which is NP-hard in general. Therefore, it's far from 
being practically useful. In a nutshell, the randomized algorithm proposed 
in this paper will overcome these drawbacks of the MW algorithm while 
retaining the throughput optimality property. For completeness, next we 
provide a brief description of the MW algorithm. 

In the wireless network model, the MW algorithm chooses a schedule 
cr(r) G X(G) every time step r G Z4. as followo 

cr(r) G arg max Q(r) • p. 

p62(G) 

In other words, the algorithm changes its decision once in unit time utilizing 
the information Q(t). The maximum weight property allows one to estab- 
lish positive recurrence by means of Lyapunov drift criteria (see Lemma 
[6]) when the arrival rate is admissible, i.e. A G A° . However, as indicated 
above picking such a schedule every time is computationally burdensome. 

^Here and everywhere else, we use notation u ■ v = X^iLi ^'■''^i fo'' ^iiy d-dimensional 
vectors u, v G R''. That is, Q(r) ■ p = J2i Qi{T) ■ Pi- 
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A natural generalization of this, called MW-/ algorithm, that uses weight 
f{Qi{-)) instead of Qi{-) for an increasing non-negative function / also leads 
to throughput optimality (cf . see [30l EH |28] ) . 

For the buffered circuit switched network model, the MW algorithm is not 
applicable. To understand this, consider the following. The MW algorithm 
would require the network to schedule active flows as z{t) G X where 

z{t) G argmaxQ(T) • z. 

This will require the algorithm to possibly preempt some of active flows 
without the completion of their service requirement. And this is not allowed 
in this model. 

3. Main Result: Simple & Efficient Randomized Algorithms. 

As stated above, the MW algorithm is not practical for wireless network 
and is not applicable to circuit switched network. However, it has the de- 
sirable throughput optimality property. As the main result of this paper, 
we provide a simple, randomized algorithm that is applicable to both wire- 
less and circuit switched network as well as it's throughput optimal. The 
algorithm requires each node (or queue) to perform only a few logical op- 
erations at each time step, it's distributed and effectively it 'simulates' the 
MW-/ algorithm for an appropriate choice of /. In that sense, it's a simple, 
randomized, distributed implementation of the MW algorithm. 

In what follows, we shall describe algorithms for wireless network and 
buffered circuit switched network respectively. We will state their through- 
put optimality property. While these algorithms seem different, philosoph- 
ically they are very similar ~ also, witnessed in the commonality in their 
proofs. 

3.1. Algorithm for Wireless Network. Let t G M+ denote the time index 
and W{t) = \Wi{t)\ G be the vector of weights at the n queues. The 
W{t) will be a function of Q(t) to be determined later. In a nutshell, the 
algorithm described below will choose a schedule (T(t) G '^{G) so that the 
weight, 'W{t) ■ cr{t), is as large as possible. 

The algorithm is randomized and asynchronous. Each node (or queue) 
has an independent Exponential clock of rate 1 (i.e. Poisson process of rate 
1). Let the kth tick of the clock of node i happen at time T^; Tq = for 
all i. By definition TJ^_^_i — T^-,k > 0, are i.i.d. mean 1 Exponential random 
variables. Each node changes its scheduling decision only at its clock ticks. 
That is, for node i the cTj(t) remains constant for t G {T^,T^_^_-^]. Clearly, 
with probability 1 no two clock ticks across nodes happen at the same time. 
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Initially, we assume that crj(O) = for all i. The node i at the fcth clock 
tick, t = T^, listens to the medium and does the following: 

o If any neighbor of i is transmitting, i.e. aj (t) = 1 for some j G A/'(i) = 
if : e E}, then set ai{t+) = 0. 



o Else, set 

ai{r 



with probability ^"^^^^^^ 
otherwise. 



Here, we assume that if (Tj(i) = 1, then node i will always transmit data 
irrespective of the value of Qi{t) so that the neighbors of node i can infer 
crj(t) by listening to the medium. 

3.1.1. Throughout Optimality. The above described algorithm for wire- 
less network is throughput optimal for an appropriate choice of weight W{t). 
Define weight Wi{t) at node i in the algorithm for wireless network as 
(3) 

W,{t) = max [fiQiilt])), ^/(QmaxdtJ))} , 

wher^ /(x) = loglog(x + e) and Qma,x{-) = niaxjQj(-). The non-local in- 
formation of Qmax(L^J)) can be replaced by its approximate estimation that 
can computed through a very simple distributed algorithm. This does not 
alter the throughput optimality property of the algorithm. A discussion is 
provided in Section [H We state the following property of the algorithm. 

Theorem 1 Suppose the algorithm of Section \3.1\ uses the weight as per 
([3]). Then, for any A G A° , the network Markov process is positive Harris 
recurrent. 

In this paper. Theorem [1] (as well as Theorem [2|) is established for the 
choice of /(x) = loglog(x + e). However, the proof technique of this paper 
extends naturally for any choice of / : M+ — t- M+ that satisfies the follow- 
ing conditions: /(O) = 0, / is a monotonically strictly increasing function, 
lim^j^oo fix) = oo and 

Jim exp(/(x)) /'(/-^5/(x))) = 0, for any S G (0, 1). 
Examples of such functions includes: /(x) = e{x) log(x + 1), where e(0) = 1, 



e{x) is monotonically decreasing function to as x — )• oo; /(x) = Y^log(x + 1); 
/(x) = logloglog(x + e*^), etc. 



^Unless stated otherwise, here and everywhere else the log(-) is natural logarithm, i.e. 
base e. 
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3.2. Algorithm for Buffered Circuit Switched Network. In a buffered cir- 
cuit switched network, the scheduhng algorithm decided when each of the 
ingress node (or queue) should request the network for availability of re- 
sources (links) along its route and upon positive response from the network, 
it acquires the resources. Our algorithm to make such a decision at each 
node is described as follows: 

o Each ingress node of a route, say Ri, generates request as per a time 
varying Poisson process whose rate at time t is equal to exp(Wi{t)). 

o If the request generated by an ingress node of route, say Ri, is accepted, 
a flow from the head of its queue leaves the queue and acquire the 
resources in the network. Else, do nothing. 

In above, like the algorithm for wireless network we assume that if the 
request of ingress node i is accepted, a new flow will acquire resources in 
the network along its route. This is irrespective of whether queue is empty 
or not - if queue is empty, a dummy flow is generated. This is merely for 
technical reasons. 

3.2.1. Throughput Optimality. We describe a specific choice of weight 
W{t) for which the algorithm for circuit switched network as described 
above is throughput optimal. Specifically, for route Ri its weight at time t 
is defined as 
(4) 

W,{t) = max [fiQiim, V/(Qmax(W))} , 

where f{x) = loglog(2; + e). The remark about distributed estimation of 
Qmax(L^J)) after ([3]) applies here as well. We state the following property of 
the algorithm. 

Theorem 2 Suppose the algorithm of Section \3.S\ uses the weight as per 
@. Then, for any A G A"^, the network Markov process is positive Harris 
recurrent. 

4. Technical Preliminaries. 

4.1. Finite State Markov Chain. Consider a discrete-time, time homo- 
geneous Markov chain over a finite state space Let its probability tran- 
sition matrix be P = [Pij] G j^l^l^l^l^ jf p irreducible and aperiodic, 
then the Markov chain is known to have a unique stationary distribution 
vr = [vTj] G mI^^' and it is ergodic, i.e. 

lim PJ^ — )• VTj, for any i,j G Q.. 
r— >c» 
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The adjoint of P, also known as the time-reversal of P, denoted by P* is 
defined as follows: 

(5) TTiP*j = TTjPji, for any ij£n. 

By definition, P* has vr as its stationary distribution as well. If P = P* then 
P is called reversible or time reversible. 

Similar notions can be defined for a continuous time Markov proces over 
Q. To this end, let P{s, t) = [Pij{s, t)] G denote its transition matrix 

over time interval [s,t]. The Markov process is called time- homogeneous if 
P{s,t) is stationary, i.e. P{s,t) = P{0,t — s) for all < s < t and is 
called reversible if P{s,t) is reversible for all < s < t. Further, if P{0,t) is 
irreducible and aperiodic for all t > 0, then this time-homogeneous reversible 
Markov process has a unique stationary distribution vr and it is ergodic, i.e. 

lim Pji{0,t) — )■ VTj, for any i,j £ Q. 

4.2. Mixing Time of Markov Chain. Given an ergodic finite state Markov 
chain, the distribution at time r converge to the stationary distribution 
starting from any initial condition as described above. We will need quan- 
titative bounds on the time it takes for them to reach "close" to their sta- 
tionary distribution. This time to reach stationarity is known as the mixing 
time of the Markov chain. Here we introduce necessary preliminaries related 
to this notion. We refer an interested reader to survey papers \17\ 124] . We 
start with the definition of distances between probability distributions. 

Definition 2 (Distance of measures) Given two probability distributions v 
and ^ on a finite space Q., we define the following two distances. The total 
variation distance, denoted 

The distance, denoted as 



More generally, for any two vectors u, v G Mlj^', we define 

II l|2 2 
I|v|l2,u = Z^UiVi. 



\2.ii. 



IS 
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We make note of the following relation between the two distances defined 

above: using the Cauchy-Schwarz inequality, we have 

(6) 



V 

1 



> 2 ||Z/ — [l^'jy 

2,M 



Next, we define a matrix norm that will be useful in determining the rate of 
convergence or the mixing time of a finite-state Markov chain. 



Definition 3 (Matrix norm) Consider a \^\ x non-negative valued 
matrix A G a given vector u S r'^'. Then, the matrix norm of 

A with respect to u is defined as follows: 



l^llu = sup ——r , 

v:EuM=0 l|V||2,u 



where Eu[v] = UiV, 



It can be easily checked that the above definition of matrix norm satisfies 
the following properties. 

PI. For matrices A,B e Rf^""^^^ and vr G m)^^ 

\\A + B\\^ < \\A\\^ + \\B\\^. 
P2. For matrix A G m[J^''''^', vr G m)^^ and c G R, 

llc^ll^ = |c|P||^. 

P3. Let A and B be transition matrices of reversible Markov chains, i.e. 
A = A* and B = B* . Let both of them have vr as their unique station- 
ary distribution. Then, 

1 1 1 1 1 1 ^A. 1 1 71" 1 1 1 1 TT • 

P4. Let A be the transition matrix of a reversible Markov chain, i.e. A = 
A*. Then, 

ll^ll ^ ''^max) 

where Amax = max{|A| 7^ 1 : A is an eigenvalue of A}. 

For a probability matrix P, we will mostly be interested in the matrix norm 
of P with respect to its stationary distribution tt, i.e. H-PUtt- Therefore, in this 
paper if we use a matrix norm for a probability matrix without mentioning 



RANDOMIZED NETWORK SCHEDULING 



13 



the reference measure, then it is with respect to the stationary distribution. 
That is, in the above example ||P|| will mean ||P||7r. 

With these definitions, it follows that for any distribution /i on 
(7) 





< ||P*|| 


^-1 




vr 


2,7r 


vr 


2,vr 



since ~ l] = 0) where ^ = [/x(z)/7r(i)]. The Markov chain of our inter- 

est, Glauber dynamics, is reversible i.e. P = P* . Therefore, for a reversible 
Markov chain starting with initial distribution /u(0), the distribution /^(t) 
at time r is such that 



(8) 



TT 



< iiPir 



2,7r 



TT 



2,7r 



Now starting from any state i, i.e. probability distribution with unit mass 



on state i, the initial distance 



1 



2,n 



in the worst case is bounded 



above by yT/vrJ^ where vTmin = min, vTj. Therefore, for any 6 > we have 



2,7r 



T > 



< 6 for any r such thai] 



log l/TTmin + log 1 A 

logl/||P|| 



e 



log l/vrmin + log l/S 



\p\\ 



This suggests that the "mixing time", i.e. time to reach (close to) the 
stationary distribution of the Markov chain scales inversely with 1 — ||P||. 
Therefore, we will define the "mixing time" of a Markov chain with transition 
matrix P as 1/(1 - ||P||). 

4.3. Glauber Dynamics & Algorithm for Wireless Network. We will de- 
scribe the relation between the algorithm for wireless network (cf. Section 
13. ip and a specific irreducible, aperiodic, reversible Markov chain on the 
space of independent sets I{G) or schedules for wireless network with graph 
G = (V, E). It is also known as the Glauber dynamics, which is used by the 
standard Metropolis-Hastings \21\ [T3] sampling mechanismthat is described 
next. 

^Throughout this paper, we shall utilize the standard order-notations: for two functions 
g,f : R+ ^ R+, g(x) = uj{f{x)) means limini^^^ g{x)/ f{x) = oo; g{x) = ^l{fix)) 
means lim inf^^^oo g(a::)//(a::) > 0; g{x) — 0{f{x)) means < liminfa;_>oo S'(a;)//(a:) < 
limsup^^^Si(2:)//(x) < oo; g{x) = 0{f{x)) means lim sup^^^ s(a;)//(a;) < oo; g{x) = 
o{f{x)) means limsup^^^ g(a;)//(a;) = 0. 
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4.3.1. Glauber Dyanmics & Its Mixing Time. We shall start off with the 
definition of the Glauber dynamics followed by a useful bound on its mixing 
time. 

Definition 4 (Glauber dynamics) Consider a graph G = (V, E) of n = 

\V\ nodes with node weights W = [Wi] G M.^. The Glauber dynamics based 
on weight W, denoted by GD(W), is a Markov chain on the space of inde- 
pendent sets ofG, T{G). The transitions of this Markov chain are described 
next. Suppose the Markov chain is currently in the state a € T^{G). Then, 
the next state, say a' is decided as follows: pick a node i £ V uniformly at 
random and 

o set a'j = aj for j / i, 

° if = for all k G M{i), then set 



a' 



1 with probability ^"^^j^'^^^) 
otherwise. 



o else set a': = Q. 



It can be verified that the Glauber dynamics GD{W) is reversible with 
stationary distribution vr given by 

(9) VTcr oc e-K'p{W ■ cr), for any a G T{G). 

Now we describe bound on the mixing time of Glauber dynamics. 

Lemma 3 Let P be the transition matrix of the Glauber dynamics GD{W) 
with n nodes. Then, 

(10) lli^ll < 1 



(11) 



< 1 



„222n+3 exp (2(n + l)VF'n 

1 



„22n+4exp(2(n + 1)VF„ 
Proof. By the property P4 of the matrix norm and Cheeger's inequality 

$2 



[B El O El [29], it is well known that ||P|| < A^ax < 1 - ¥ ^^^^^ ^ ^he 



conductance of P, defined as 

Q{S, 5= 



$ = min 



5c2{G):7r{S)<i vr(S')7r(S''=) ■ 
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where = I{G)\S, Q{S, S^) = E^G5,<T'eS- '^O- Now we have 



<l> > mill Q(5,5^) 

SCX{G) 

> mill -K(cr)P(a, a') 

P(<t,<t')^0 



1 

> VTmin • mm 



I 



> 



> 



n 1 + exp(Wj' 
1 1 



1 



2'^exp(nVF„ 



n2"+iexp((n + l)T^ma.)' 



Therefore 



IPII < 1 



„222n+3 exp (2(n + ■ 
Now consider e'^^^^'^^~^\ Using properties PI, P2 and P3 of matrix norm, 



we have: 



k\ 



< 

fc=0 
n{\\P\\-l) 



fc=0 

n'^ IIPI 



(12) 



= e 
< 1 



n(l 



A;! 



IPII 



In the last inequahty, we have used the fact that ||P|| < 1 and e ^ < 1 — x/2 
for ah X G [0, 1]. Hence, from the bound of ||P||, we obtain 



(13) 



.n{P-I) 



< 1 



n22«+4exp(2(n + l)Wn,ax)' 
This completes the proof of Lemma [3l 



□ 



4.3.2. Relation to Algorithm. Now we relate our algorithm for wireless 
network scheduling described in Section 13.11 with an appropriate continuous 
time version of the Glauber dynamics with time-varying weights. Recall 
that Q(i) and CT{t) denote the queue-size vector and schedule at time t. The 
algorithm changes its scheduling decision, (T{t), when a node's exponential 
clock of rate 1 ticks. Due to memoryless property of exponential distribution 
and independence of clocks of all nodes, this is equivalent to having a global 
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exponential clock of rate n and upon clock tick one of the n nodes gets 
chosen. This node decides its transition as explained in Section lSJl Thus, the 
effective dynamics of the algorithm upon a global clock tick is such that the 
schedule cr{t) evolves exactly as per the Glauber dynamics GD{W{t)). Here 
recall that W{t) is determined based on Q([tJ). With abuse of notation, let 
the transition matrix of this Glauber dynamics be denoted by GD{W{t)). 
Now consider any r € Z+. Let Q(r),<T(r) be the states at time r. Then, 



E 



V(r+1) 



Q(r),a(r)j = J2^-(r)^<C = k)GD{WiT)Y 



k=0 



where we have used notation So- for the distribution with singleton support 
{<t} and C is a Poisson random variable of mean n. In above, the expecta- 
tion is taken with respect to the distribution of <t(t + 1) given Q(T),<T(r). 
Therefore, it follows that 



E 



^<t(t+1) 



Q(T),a(r) 



V(r) 



MGD{W{r))-I) 



(14) = S^^,^P{t) 

where P(t) = e"(G'l5(W(r))-/)_ general, for any 5 £ [0, 1] 



(15) 



E 



'(T(r+<5) 



Q(r),«T(T) 



where P^{t) = e 



5niGD{WiT))~I) 



4.4. Loss Network & Algorithm for Circuit Switched Network. For the 
buffered circuit switched network, the Markov chain of interest is related to 
the classical stochastic loss network model. This model has been popularly 
utilized to study the performance of various systems including the telephone 
networks, human resource allocation, etc. (cf. see [E]). The stochastic loss 
network model is very similar to the model of the buffered circuit switched 
network with the only difference that it does not have any buffers at the 
ingress nodes. 



4.4.1. Loss Network & Its Mixing Time. A loss network is described 
by a network graph G = (y,E) with capacitated links [Cgjegs, n routes 
{Ri : Ri G E,l < i < n} and without any buffer or queues at the ingress of 
each route. For each route Ri, there is a dedicated exogenous, independent 
Poisson arrival process with rate (pi. Let Zi{t) be number of active flows on 
route i at time t, with notation z{t) = [zi{t)]. Clearly, z(t) E X due to 
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network capacity constraints. At time t when a new exogenous flow arrives 
on route Ri, if it can be accepted by tlie network, i.e. z(t) + ej € X, then it is 
accepted with Zi{t) — )• Zi{t)+1. Or else, it is dropped (and hence lost forever). 
Each flow holds unit amount of capacity on all links along its route for time 
that is distributed as Exponential distribution with mean 1, independent of 
everything else. Upon the completion of holding time, the flow departs and 
frees unit capacity on all links of its own route. 

Therefore, effectively this loss network model can be described as a finite 
state Markov process with state space X. Given state z = [zj] G X, the 
possible transitions and corresponding rates are given as 
(16) 

with rate (pi if z + Ci £ X, 
with rate Xj. 

It can be verified that this Markov process is irreducible, aperiodic, and 
time-reversible. Therefore, it is positive recurrent (due to the finite state 
space) and has a unique stationary distribution. Its stationary distribution 
vr is known (cf. [E]) to have the following product-form: for any z £ X, 




(17) vr, oc n 



1=1 

We will be interested in the discrete-time (or embedded) version of this 
Markov processes, which can be defined as follows. 

Definition 5 (Loss Network) A loss network Markov chain with capac- 
itated graph G = iV^E), capacities Ce,e € E and n routes Ri, 1 < i < n, 
denoted by LN{cf)) is a Markov chain on X . The transition probabilities of 
this Markov chain are described next. Given a current state z £ X, the next 
state z* £ X is decided by first picking a route Ri uniformly at random and 
performing the following: 

o z* = Zj for j i and z* is decided by 

Zi + 1 with probability ^ • l^z+eteX} 
Zi — 1 with probability ^ , 
^ Zi otherwise. 

where TZ = Yl.i(pi + Cmax- 

LN(0) has the same stationary distribution as in (jl7p . and it is also irre- 
ducible, aperiodic, and reversible. Next, we state a bound on the mixing 
time of the loss network Markov chain LN(0) as follows. 
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Lemma 4 Let P be the transition matrix of LN{(f)) with n routes. If 4> 
exp(Vt^) with% Wj > for all i, then, 

1 

(18) ||P|| < 1 



(19)" 



gn7^(P-/) 



1 

< 1 



Proof. Similarly as the proof of Lemma [3l a simple lower bound for the 
conductance <1> of P is given by 

(20) <^ > VTmin- min P^,^/. 

To obtain the lower bound of VTmini recall the equation (|17p . 

-Inf. 

where Z = X^zgA' TYi=i TT' ^'^^ consider the following: 
and 



> ^ > 



By combining the above inequalities, we obtain 

(21) 7r„,in > ^ 



C^^^— exp(nC„,axWmax) ' 

On the other hand, one can bound minp^ ^,^o -f^.z' £is follows: 

1111 1 
(22)P,,,, > - . > > 



n TZ n n(/)max + Cmax 2ri2Cmax exp(Wmax) ' 

where we use the fact that x + y < 2xy if x,y > 1. Now, by combining ([21 
and ([22]) . we have 

1 

$ > 



2n2CS,S'-+"+i exp ((nC^^ax + l)Wmax) ■ 



^We use the following notation: given a function g : R —!> R and a d-dimensional vector 
u G R'', let g{u) = [gim)] € R'*. 
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Therefore, using the property P4 of the matrix norm and Cheeger's inequal- 
ity, we obtain the desired conclusion as 



l^'ll < A, 



$2 

< 1 < 



Furthermore, using this bound and similar arguments in the proof of Lemma 
m we have 



^nn{P-I) 



< 1 



16n3C^"^— exp {2{nCm^ + l)Wn 



□ 



4.4.2. Relation to Algorithm. The scheduling algorithm for buffered cir- 
cuit switched network described in Section [3r2] effectivelv simulates a stochas- 
tic loss network with time- varying arrival rates cf){t) where = exp(Wi{t)) 
That is, the relation of the algorithm in Sectioning] with loss network is sim- 
ilar to the relation of the algorithm in Section 13.11 with Glauber dynamics 
that we explained in the previous section. To this end, for a given r G let 
Q(r) and z{t) be queue-size vector and active flows at time r. With abuse of 
notation, let LN {exp{W (t))) be the transition matrix of the corresponding 
Loss Network with W{t) dependent on Q(t). Then, for any 6 € [0, 1] 

(23) 

,n<57^(r)(LA^(exp(V^(r)))-7) 

where 7^(r) = ^ . exp(Wi(r)) + C^^,. 

4.4.3. Positive Harris Recurrence & Rs Implication. For completeness, 
we define the well known notion of positive Harris recurrence (e.g. see [21 [3]). 
We also state its useful implications to explain its desirability. In this paper, 
we will be concerned with discrete-time, time-homogeneous Markov process 
or chain evolving over a complete, separable metric space X. Let Bx denote 
the Borel c-algebra on X. We assume that the space X be endowed with a 
norni^l, denoted by | • |. Let X[t) denote the state of Markov chain at time 
r G Z+. 

Consider any A G Bx- Define stopping time Ta = inf{r > 1 : X{t) G A\. 
Then the set A is called Harris recurrent if 



E 



Q(r),2(r) 



Vi^{Ta < oo) = 1 for any x G X, 



®One may assume it to be induced by the metric of X, denoted by d. For example, for 
any x G X, jx| — d{0, x) with respect to a fixed G X. 
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where Prx(-) = Pr(-|X(0) = x). A Markov chain is called Harris recurrent if 
there exists a a-finite measure fi on (X,i3x) such that whenever fi{A) > for 
A G i3x, A is Harris recurrent. It is well known that if X is Harris recurrent 
then an essentially unique invariant measure exists (e.g. see Getoor [llj). If 
the invariant measure is finite, then it may be normalized to obtain a unique 
invariant probability measure (or stationary probability distribution); in this 
case X is called positive Harris recurrent. 

Now we describe a useful implication of positive Harris recurrence. Let vr 
be the unique invariant (or stationary) probability distribution of the posi- 
tive Harris recurrent Markov chain X. Then the following ergodic property 
is satisfied: for any x G X and non- negative measurable function / : X — ?• M_|_, 

1 

lim — /(X(r)) ^n[f], Prx-almost surely. 

r=0 

Here E7r[/] = / f{z)-K{z). Note that ]E^[/] may not be finite. 

AAA. Criteria for Positive Harris Recurrence. Here we introduce a well 
known criteria for establishing the positive Harris recurrence based on exis- 
tence of a Lyapunov function and an appropriate petit set. 

We will need some definitions to begin with. Given a probability distri- 
bution (also called sampling distribution) a on N, the a-sampled transition 
matrix of the Markov chain, denoted by Ka is defined as 

Ka{-K, B)=Y^ a{T)P^{-K, B), for any x G X, 5 G Bx- 

T>0 

Now, we define a notion of a petite set. A non-empty set A G Bx is called 
lia-petite if /i^ is a non-trivial measure on (X,i3x) and a is a probability 
distribution on N such that for any x G A, 

i^a(x,-) >/^a(-)- 

A set is called a petite set if it is /ia-petite for some such non-trivial measure 
lia. A known sufficient condition to establish positive Harris recurrence of a 
Markov chain is to establish positive Harris recurrence of closed petite sets 
as stated in the following lemma. We refer an interested reader to the book 
by Meyn and Tweedie [22] or the recent survey by Foss and Konstantopoulos 
[9] for details. 

Lemma 5 Let B he a closed petite set. Suppose B is Harris recurrent, i.e. 
Pi'x(7b < c«) = 1 for any x G X. Further, let 

sup Ex [Tb] < oo. 
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Then the Markov chain is positive Harris recurrent. Here Ex is defined with 
respect to Pix- 

Lemma [5] suggests that to establish the positive Harris recurrence of the 
network Markov chain, it is sufficient to find a closed petite set that satisfies 
the conditions of Lemma [5j To establish positive recurrence of a closed petit 
set, we shall utilize the drift criteria based on an appropriate Lyapunov 
function stated in the following Lemma (cf. [H Theorem 1]). 

Lemma 6 Let L : X — >■ IR+ he a function such that L(x) —)■ oo as |x| — )■ oo. 
For any k > 0, let B,^ = {x : L(x) < k}. And let there exist functions 
h,g : X — > Z+ such that for any x € X, 

E [L(X(5(x))) - L{X{0))\X{0) = x] < -/i(x), 

that satisfy the following conditions: 

(a) infxgx ^(x) > — oo. 

(b) liminfi(x)_^oo^(x) > 0. 

(c) supi(x)<.^ ^(x) < oo for all 7 > 0. 

(d) limsupi(x)^oo5'(x)//i(x) < 00. 

Then, there exists constant kq > so that for all kq < k, the following holds: 

(24) Ex [TbJ < 00, for anyjceX 

(25) sup Ex [TbJ < 00. 

That is, is positive recurrent. 

5. Proofs of Theorems [1] & [2], This section provides proofs of The- 
orems [U and [2j We shall start with necessary formalism followed by a sum- 
mary of the entire proof. This summary will utilize a series of Lemmas whose 
proofs will follow. 

5.1. Network Markov Process. We describe discrete time network Markov 
processes under both algorithms that we shall utilize throughout. Let r S Z+ 
be the time index. Let Q(t) = [<5i(r)] be the queue-size vector at time r, 
x{t) be the schedule at time r with x{t) = a^r) S T{G) for the wireless 
network and x{t) = z{t) G X for the circuit switched network. It can be 
checked that the tuple X{t) = (Q(r), x{t)) is the Markov state of the net- 
work for both setups. Here X{t) G X where X = M!;^ x I{G) oTX = 'L\y. X. 
Clearly, X is a Polish space endowed with the natural product topology. Let 
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Bx be the Borel cr-algebra of X with respect to this product topology. For 
any x = {Q,x) G X, we define norm of x denoted by |x| as 

|x| = IQI + |a;|, 

where |Q| denotes the standard ii norm while |a;| is defined as its index in 
{0, . . . , |0| — 1}, which is assigned arbitrarily. Since \x\ is always bounded, 
|x| — )• oo if and only if |Q| — )• oo. Theorems [1] and [2] wish to establish that 
the Markov process X(t) is positive Harris recurrent. 

5.2. Proof Plan. To establish the positive Harris recurrence of X(t), 
we will utilize the Lyapunov drift criteria to establish the positive recur- 
rence property of an appropriate petit set (cf. Lemma [5]). To establish the 
existence of such a Lyapunov function, we shall study properties of our ran- 
domized scheduling algorithms. Specifically, we shall show that in a nutshell 
our schedule algorithms are simulating the maximum weight scheduling al- 
gorithm with respect to an appropriate weight, function of the queue-size. 
This will lead to the desired Lyapunov function and a drift criteria. The de- 
tailed proof of positive Harris recurrence that follows this intuition is stated 
in four steps. We briefly give an overview of these four steps. 

To this end, recall that the randomized algorithms for wireless or circuit 
switched network are effectively asynchronous, continuous versions of the 
time-varying GD{W{t)) or LN(exp(VF(t))) respectively. Let 7r(t) be the 
stationary distribution of the Markov chain GD(W{t)) or LN(exp("H^(i))); 
^{t) be the distribution of the schedule, either cr{t) or z{t), under our algo- 
rithm at time t. In the first step, roughly speaking we argue that the weight 
of schedule sampled as per the stationary distribution 7r{t) is close to the 
weight of maximum weight schedule for both networks (with an appropri- 
ately defined weight). In the second step, roughly speaking we argue that 
indeed the distribution fj,{t) is close enough to that of 7r(t) for all time t. In 
the third step, using these two properties we establish the Lyapunov drift 
criteria for appropriately defined Lyapunov function (cf. Lemma [6|). In the 
fourth and final step, we show that this implies positive recurrence of an 
appropriate closed petit set. Therefore, due to Lemma [5] this will imply the 
positive Harris recurrence property of the network Markov process. 

5.3. Formal Proof. To this end, we are interested in establishing Lya- 
punov drift criteria (cf. Lemma [6]). For this, consider Markov process start- 
ing at time in state X{0) = (Q(0),a;(0)) and as per hypothesis of both 
Theorems, let A G (1 — e) Conv(i7) with some e > and i7 = T{G) (or X). 
Given this, we will go through four steps to prove positive Harris recurrence. 
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5.3.1. Step One. Let 7r(0) be the stationary distribution of GD{W{0)) 
or LN {exp{W (0))) . The following Lemma states that the average weight of 
schedule as per 7r(0) is essentially as good as that of the maximum weight 
schedule with respect to weight /(Q(0)). 

Lemma 7 Let x be distributed over il. as per vr(0) given Q(0). Then, 



The proof of Lemma [7] is based on the variational characterization of distri- 
bution in the exponential form. Specifically, we state the following proposi- 
tion which is a direct adaptation of the known results in literature (cf. |1U]). 



Proposition 8 Let T : — > M and let be space of all distributions 

on 0. Define F : M{n) -^R as 



where Her{p) is the standard discrete entropy of fi. Then, F is uniquely 
maximized by the distribution u, where 



where Z is the normalization constant (or partition function). Further, with 
respect to v, we have 



Proof. Observe that the definition of distribution i' implies that for any 



(26) 




0(1). 



F{fi)=K^{T{j^)) + HER{fi) 



i^x = 17 exp (T(x)) , for any x G $7 



IEi.[T(x)] > maxr(x) -log|Jl 



x G r2, 



r(x) = logZ + logZ^x- 
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Using this, for any distribution /x on il, we obtain 
= ^ ^xr(x) - ^ Mx log ^x 

X X 

= ^ ^x(log Z + log Z^x) - X] '"^ /^J< 

X X 

= ^ Mx log + ^ log ^ 

X X 

= log Z + ^ /ix log ■ 

X 

< logZ + log(^J^^x^) 
= logZ 

with equality if and only if = z/. To complete other claim of proposition, 
consider x* £ argmaxT(x). Let be Dirac distribution /ix = l[x=x*]- Then, 
for this distribution 

F{^l) = T{^*). 

But, F[i') > F(n). Also, the maximal entropy of any distribution on il. is 
log|0|. Therefore, 

T(x*) < F(i/) 

= E,[T{^)] + HER{iy) 
(27) < E^[r(x)]+log|f]|. 



Re-arrangement of terms in (j27p will imply the second claim of Proposition 
[H This completes the proof of Proposition [H □ 

Proof of Lemma The proof is based on known observations in the 
context of classical Loss networks literature (cf. see [E]). In what follows, 
for simplicity we use vr = 7r(0) for a given Q = Q(0). From ([9]) and (fT7|) . it 
follows that for both network models, the stationary distribution vr has the 
following form: for any x £ Q, 



^ jj- exp_(W^xJ ^ exp I WjXj - log(xj!) 1 . 

To apply Proposition [8l this suggest the choice of function T : Af — )• M as 
T{x) = ^ WiXi — log(2;j!), for any x £ Q. 
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Observe that for any x G Q., Xi takes one of the finitely many values in 
wireless or circuit switched network for all i. Therefore, it easily follows that 

< J]log(xJ) < 0(1), 

i 

where the constant may depend on n and the problem parameter (e.g. Cmax 
in circuit switched network). Therefore, for any a; G 0, 



T{x) < ^VFiXi 

i 

(28) < T{x) + 0{1). 

Define x = argmaxa,gn WjXj. From (f28]l and Proposition El it follows 
that 



> maxT(a;) — log 

> T{x)-iog\n\ 

= (^w^x}j -o{i)-iog\n\ 

(29) = (maxW ■ x] - 0(1). 

From the definition of weight in both algorithms and dH) for a given 
Q, weight W = [Wi] is defined as 

Wi = max |/(Qi), \/ /(Qmax)} • 

Define ri = -j ^ — n^^- To establish the proof of Lemma [71 we will consider 

4 max 33 ll-^lli 

Qmax such that it is large enough satisfying 



max ) > VfiQ 

For smaller Qmax we do not need to argue as in that case (p6|) (due to 0(1) 
term) is straightforward. Therefore, in the remainder we assume Qmax large 
enough. For this large enough Qmaxi it follows that for all i, 



(30) < Wi-f{Q,) < V/(Qmax) < ??/(Qmax) 
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Using (i30]l . for any a; G 



< 



< 



< 



W -x- f{Q)-x 
||:r||i||W-/(Q)|| 
ll^^lli X iifiQ max] 



oo 



(a) 



< 



(31) 



< 



£(max/(Q).^ 



where (a) is from our choice of ?? = i — n^^. For (b), we use the fact 

that the singleton set {i}, i.e. independent set {i} for wireless network and 
a single active on route i for circuit switched network, is a valid schedule. 
And, for i = aigmaxj Qj, it has weight /(Qmax)- Therefore, the weight of 
the maximum weighted schedule among all possible schedules in 0, is at least 
/(Qmax)- Finally, using ([29]) and (f3T]) we obtain 



This completes the proof of Lemma [71 

5.3.2. Step Two. Let fj,{t) be the distribution of schedule x{t) over at 
time t, given initial state X{0) = (Q(0), a;(0)). We wish to show that for 
any initial condition x{0) G Q, for t large (but not too large) enough, fj,{t) 
is close to 7r(0) if (5max(0) is large enough. Formal statement is as follows. 

Lemma 9 For a large enough (5max(0), 



for t £ I = [6i((5max(0)), &2(Qmax(0))], where 61,62 o-re integer-valued func- 
tions on ]R+ such that 

61, 62 = polylog (<3max(0)) and 62/61 = 6 (log (Qmax(O))) . 

In above the constants may depend on e, Cmax CLnd n. 



[/(Q) • x] 




(32) 



7r(0)||^^<e/4, 
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The notation polylog(2;) represents a positive real-valued function of z that 
scales no faster than a finite degree polynomial of log z. 

Proof of LemmalM We shall prove this Lemma for the wireless network. The 
proof of buffered circuit switch network follows in an identical manner. Hence 
we shall skip it. Therefore, we shall assume = I{G) and x{t) = cr(t). 

First, we establish the desired claim for integral times. The argument for 
non integral times will follow easily as argued near the end of this proof. For 
t = T £ Z4_, we have 



E • P(r)] 



where recall that P(r)=e"^'^^^^^'^^''~^^ and the last equality follows from 
()14p . Again recall that the expectation is with respect to the joint distribu- 
tion of {Q(r), <t(t)}. Hence, it follows that 

Mr + 1) = E [<5^(,) • P(t)] 



(a) 



E 



E 



E 



E 



S^ir)-P(.r) Q(t) 



Q(r) .P{r) 



E[/2(T).P(r) 



where 



/i(T)=/i(Q(r))^E 



Q(t) 



In above the expectation is taken with respect to the conditional marginal 
distribution of <t(t) given Q(t); (a) follows since P{t) is a function of Q(t). 
Next, we establish the relation between /^(t) and ^(r + 1). 



where e(r) = E [/i(T) 
to Q(t). Similarly, 



E[Mt)-P(t)] 

E[MT).P(0)]+E[/i(T)-(P(r)-P(0))] 
E[MT)].P(0) + e(T) 
MT)-P(0) + e(T), 

(P(r) — P(0))]. Here the expectation is with respect 
fi{T) ■ P{0) + e(T) 

(Mr-l)-P(0) + e(T-l))-P(0)+e(r) 
/i(r - 1) • P{Of + e(T - 1) • P(0) + e(r). 
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Therefore, recursively we obtain 

r 

(33) + = MO)-P(Or+^+J]e(r-s)-P(Or. 

We will choose bi (which will depend on Qmax(O)) such that for t >bi, 

(34) M0)-P{0r-7rm\^y < e/8. 

That is, hi is the mixing time of -P(O). Using inequalities ([8]), ([6|) and Lemma 
m it follows that 

^1 = bl(Qmax(0)) = polylog(Qmax(0)) . 

In above, constants may depend on n and e. Therefore, from (j33p and (j34p . 
it suffices to show that 



(35) 



T-l 



^e(r-l-s).P(0)^ 



s=0 



<e/8, 



for T E / = [61,62] with an appropriate choice of 62 = 62(Qmax(0)). To this 
end, we choose 

62 = 62(Qmax(0)) = r6llog(g^ax(0))l. 

Thus, 62 (Qmax (0) ) = polylog ((5max(0)) as Well. With this choice of 62, we 
obtain the following bound on e(T) to conclude (j35p . 

||e(r)||i = ||E[/i(r).(P(T)-P(0))]||i 
< E[||^(r).(P(T)-P(0))||i] 



(36) 



O(E[||P(T)-P(0)|U]) 

0{E[\\GW{W{T))-GW{Wm\U) 
1 1 

max 

i 



O j^E 
O ( E 



l + exp(P^i(r)) l + exp(l^i(0)) 
max|Wi(r) - Wi{0)\ 



O fmaxE[|t^,(T) - W^,(0)|]^ . 



In above, (a) follows from the standard norm inequality and the fact 
that ||A(t)||i = 1, (b) follows from Lemma [TT] in Appendix, (c) follows 
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directly from the definition of transition matrix GD{W), (d) follows from 
1-Lipschit^ property of function 1/(1 + e^) and (e) follows from the fact 
that vector W{r) being 0(1) dimensional. 
Next, we will show that for all i and r < 62, 



(37) 



E[\WiiT)-WiiO)\] 



o 



superpolylog (Qmax(O)) 



the notation superpolylog(z) represents a positive real- valued function of z 
that scales faster than any finite degree polynomial of log z. This is enough 
to conclude ([35]) (hence complete the proof of Lemma [9|) since 



T-1 



s=0 



T-1 



< ^||e(T-l-.)-P(0)^||i 

s=0 

T-1 

= ^0(||e(r-l-.)||J 
r 

superpolylog (Qmax(O)) 





s=0 


(a) 




(b) 


e 


< 






4' 



where we use ([36]) and ([37]) to obtain (a), (b) holds for large enough 
and T < 62 = polylog ((5max(0)). 



c(0) 



Now to complete the proof, we only need to establish (|37p . This is the 
step that utilizes 'slowly varying' property of function f{x) = log log(a; + e). 
First, we provide an intuitive sketch of the argument. Somewhat involved 
details will be follow. To explain the intuition behind (|37p . let us consider 
a simpler situation where i is such that Qi{0) = (5max(0) and f{Qi{T)) > 
y^fiQm^Ar)) for a given r G [0,62]. That is, let T^i(r) = /(Q»(r)). Now, 



■^A function / : R ^ R is it-Lipschitz if |/(s) - f{t)\ < k\s - t\ for all s,t £ R. 

*We note here that the O(-) notation means existences of constants that do not depend 
scaling quantities such as time r and Q(0); however it may depend on the fixed system 
parameters such as number of queues. The use of this terminology is to retain the clarity 
of exposition. 
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consider following sequence of inequalities: 



\Wi{T)-Wim 



\f{Q^{r)-f{Q^m\ 



(a) 



< f'{C)\Qi{r)-Qi 



for some C around Qi{0) 



(b) 

< /'(min{Q,(r),Q,(0)})O(r) 

< /'(Q,(0) - 0(T))0(r) 



(38) 



id) 



o 



Qm 



In above, (a) follows from the mean value theorem; (b) follows from mono- 
tonicity of /' and Lipschitz property of Qi{-) (as a function of r) - which 
holds deterministically for wireless network and probabilistically for circuit 
switched network; (c) uses the same Lipschitz property; and (d) uses the 
fact that r < 62 and 62 = polylog(Qmax(0)), (5max(0) = Qi(0). Therefore, 
effectively the bound of (f38|) is O(l/superpolylog((5max(0)). 

The above explains the gist of the argument that is to follow. However, 
to make it precise, we will need to provide lots more details. Toward this, 
we consider the following two cases: (i) /(Qi(0)) > \/ /(Qmax(O)), and (ii) 
f{Qi{0)) < \J /(Qmax(O)). In what follows, we provide detailed arguments 
for (i). The arguments for case (ii) are similar in spirit and will be provided 
later in the proof. 



Case (i): Consider an i such that f{Qi{0)) > \J /('3max(0)). Then, 

E[|Tyi(T)-I^,(0)|] 
= E[|W,(r)-/(gi(0))|] 



(39) 



{/((3.M)<V/(Qmax(T))} 
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where each equahty follows from ([3]) . The first term in (139p can be bounded 
as follows 



E 



<E[|/(Q,(T))-/(Qi(0))|] 

< E [/' (min{Q,(r), Qi{0)}) \Qi{r) - Q 

(a) 

< ^ E 



f'{mm{Q,{T),QM}f ' [{Qi{r) - QiiOW] 

0{t) 



+ e 



< 



\ 



f f^/-^(V/(Qmax(0)))) +e ^ 

^2 ^ V v/-^(/ 



/(Qmax(O)) 



0(r) 



(40) O 



superpolylog (Qmax(O)) 



In above, (o) follows from concavity of /. For (a) we use the standard 
Cauchy-Schwarz inequality K[XY] < ^E[X2]^E[y2]. For (b), note that 
given Qi{0), E[[Qi(0) - Qi(r)]2] = ©(r^) for both network models - for 
wireless network, it is deterministically true due to Lipschitz property of 
Q(-); for circuit switched network, it is due to the fact that the arrival as 
well as (the overall) departure processes are bounded rate Poisson processes. 
Given this, using Markov's inequality it follows that 



Pr (^min{Q,(T),Q,(0)}<^ 



O 



Q^{0) 



Finally, using the fact that supygjj^ fiv) = 0(1), we obtain (b). Now (c) 
follows from the condition of Qi(0) that /(Qi(0)) > \//(Qmax(0)). And, (d) 
is implied by r < 62 = polylog(Qmax(0)), f{x) = loglog(x + e). 
Next, we bound the second term in (1391) . We will use notation 



Mr) = {f{Q^{r)) < \//(gmax(r)) & V/(Qmax(r)) > /(Qi(0))} 
B{t) = {/(Q.(r)) < v//(Qmax(T))& ^/(Qmax(r)) < f{Q^{0))} 
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Then, 



E 



v//(Qmax(T)) - fiQm) 

E 



I 



{f{Q^ir))<^/f{QmMr))} 



+ IE [(/(Q./(0)) - v//(Qmax(r))) -Ib 



J(r) 



(a) 
< E 



(41) 



^/(Qmax(T)) - x//(Qmax(0))j • I^Cr) 

+ E[(/(Qi(0))-/(Qi(r))).Is(,)] 

< E [|/(Qn,ax(T)) - /(Qmax(0))|] + E [|/(Q.(0)) - /(Q,(t) 
1 



superpolylog (Qmax(O)) 



In above, (a) follows because we are considering case (i) with /(Qj(0)) > 
\/ /(Qmax(O)) and definition of event B{t); (b) follows from 1-Lipschitz prop- 
erty of y/^ function and appropriate removal of indicator random variables. 
For the final conclusion, we observe that the arguments used to establish (|40p 
imply the O(l/superpolylog((5max(0))) bound on both the terms in very sim- 
ilar manner: for the term corresponding to |/(Qmax(''")) — /(Qmax(0))|, one 
has to adapt arguments of MOD by essentially replacing the index i by max. 
This concludes the proof of ([37]) for case (i) of f{Qi{0)) > ^ /(Qmax(O)). 



Case (a): Now consider i such that f{Qi{0)) < \J /((5max(0)). Then, 



E[|Wi(r)-VFi(0)|] =E Wi{T) - V/(Qmax(0)) 



(42) 



E 
E 



/(Q.(r)) - V/(Qmax(0)) 



v'/(Qmax(r)) - V/(Qmax(0)) 



{/(Q»{T))>V/((3max{T))} 
I 



{/{Q.(r))<V/(Qmax(r))} 



First observe that by 1-Lipschitz property of function, the second term 
can be bounded as (similar to (jlT]) ) 



E 



V/(Qmax(r)) - v'/(Qmax(0)) 



I 



{/(Q.{r))<V/(0"->=(^))} 



<]E[|/(Q^ax(r))-/(Qmax(0))|] 



(43) 



^ ' superpolylog (Qmax(O)) 
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Therefore, we are left with proving the first term of (j42p . We will follow 
similar line of arguments as those used for ()4ip . Define 

A'{r) = {/(Q.(t)) > x//(Qmax(r))& V/(Qmax(0)) > f{Q^{T))] , 
fmr)) > V/(Qmax(r))& V/(Qmax(0)) < /(Q.(r))} . 



i?'(r) 
Then, 



E 



/(Q^(t)) - V/(Qmax(0)) 

: E 



I 

V/(Qmax(0))-/(g.(r))) 

+ E 



{/(Q»(r))>V/{Qmax(r))} 



(a) 
< E 



- V/(Qmax(0))j -I^M 
V/(Qmax(0)) - V/(Qmax(r))) • I^(,) 



+ E 



/(Q^W) - \//(Qmax(0)) -I^M 



(fe) 

< o 



superpolylog((5max(0)) 



+ E 



V/(Qmax(0)))-lB(,) 



(44) 



In above, (a) follows because we are considering case (i) with f{Qi{T)) > 
\/ fiQmaxiT)) and definition of event B{t); (b) follows from 1-Lipschitz prop- 
erty of -y/^ function and appropriate removal of indicator random variables 
as follows: 



E 



(45) 



\//(Qmax(0)) - v'/(Qmax(r))j • 
<E[|/(Q^ax(r))-/(Qmax(0))|] 

= O 



1 



superpolylog((5max(0)). 
Finally, to complete the proof of case (ii) using (j42p , we wish to establish 

1 



E 
(46) 



f{Q^{r)) - ^/(Qmax(O)) • I^M 



o 



superpolylog ((5max(0)) 
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Now suppose X G M+ be such that f{x) = a/ /((3max(0)- Then, 

E [(/(Q.(-r)) - V/(Qmax(0))) • 
= E[(/(Q,(r))-/(x))-lBM] 

< E[r(x)(Q,(r)-x).lB(,)] 
= /'(x)E[(Q,(r)-x).lB(,)] 
(b) 

< /'(x) E[(Q,(t)-Q,(0))-Ibm] 
</'(x)E[|Q,(r)-Q, 

= /'(^) O (r) 

(47) O ^ ^ 



superpolylog (Qmax(O)) 



In above, (a) follows from concavity of /; (b) from (5i(0) < x and Qi{T) > x 
implied by case (ii) and B'{t) respectively; (c) follows from arguments used 
earlier that for any i, E[(Qi(T) - Qi{Q)f] = 0{t'^)] (d) follows from r < 
62 = polylog(Qmax(0)) and 



f'(x) = { ^ ) 

Vsuperpolylog(Qmax(0))y 



This complete the proof of ()37p for both cases and the proof of Lemma [9] for 
integral time steps. A final remark validity of this result about non-integral 
times is in order. 

Consider t & I and t ^ Z+. Let r = [t\ and t = r + 5 for 5 G (0, 1). Then, 
it follows that (using formal definition as in (jlSp ) 

ix{t) = I,{t + 5) = ^l{T)P\Q)+¥.[fi{T){P'{T)-P'm 

(48) = ^l{l^)P{QYP\<^) + e{T + 5). 

Now it can be checked that P^(0) is a probability matrix and has 7r(0) as 
its stationary distribution for any 5 > 0; and we have argued that for r large 
enough /x(0)P(0)^ is close to 7r(0). Therefore, /i(0)P(0)^P'^(0) is also equally 
close to '/r(O). For e(r + 5), it can be easily argued that the bound obtained 
in (|36p for e(r + 1) will dominate the bound for e(r + 5). Therefore, the 
statement of Lemma holds for any non-integral t as well. This complete the 
proof of Lemma [9l 
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5.3.3. Step Three: Wireless Network. In this section, we prove Lemma [6] 
for the wireless network model. For Markov process X{t) = (Q(t), <T(t)), we 
consider Lyapunov function 

L{x{t)) = Y,F{Q^m 

i 

where F{x) = Jq f{y) dy and recall that f{x) = loglog(x + e). For this 
Lyapunov function, it suffices to find appropriate functions h and g as per 
Lemma El for a large enough (5max(0). Therefore, we assume that (5max(0) 
is large enough so that it satisfies the conditions of Lemma [9l To this end, 
from Lemma [9l we have that for t € I, 

|E.(o)[/(Q(0))-*T]-E^(i)[/(Q(0))-^]| < |(^max^/(Q(0))-p). 

Thus from Lemma [71 it follows that 
(49) 

Now we can bound the difference between L(X{t + 1)) and L(X{t)) as 
follows. 

L{X{t + 1)) - L{X{t)) = (F(Q(r + 1)) - F(Q(t))) • 1 

< /(Q(r + 1)).(Q(T + 1)-Q(r)), 

< /(Q(r)).(Q(T + l)-Q(r))+n, 

where the first inequality is from the convexity of F and the last inequality 
follows from the fact that f{Q) is 1-Lipschitz. Therefore, 

L(X(r + 1)) - L(X(t)) = (F(Q(t + 1)) - F(Q(r))) • 1 

rr+l 



< /(Q(r)) • (^A{t, T + l)- cr(y)l{Q^(^)>o} dy^ + n 

< /(Q(r)) • A(t, t + 1)- j^^ f{Q{y)) ■ (T(y)l|Q^(j,)>o} dy + 2n 
(50) = /(Q(r)) • Air, r + 1) - j^'"' /(Q(y)) • o-(y) dy + 2n, 



36 



SHAH & SHIN 



where again (a) follows from the fact that f{Q) is 1-Lipschitz. Given initial 
state X{0) = X, taking the expectation of ([50]) for r, r + 1 G I, 

Ex[L(X(r + l))-L(X(r))] 

< Ex[/(Q(r)) • Air, r + 1)] - IEx[/(Q(y)) • ^(y)] + 2n 

= Ex[/(Q(r)).A]-^"^'E,[/(Q(y)).a(y)]dy + 2n, 

where the last equality follows from the independence between Q(r) and 
A{t,t + 1) (recall, Bernoulli arrival process). Therefore, 

Ex[L(X(t + 1))-L(X(t))] 

< Ex[/(Q(r)) • A] - £^ E^[f{Qm • cT{y)] dy 

- J^^' ]Ex[(/(Q(y)) - /(Q(0))) • <T{y)] dy + 2n 

< /(Q(0) + r • 1) • A - 1^^' Ex[/(Q(0)) • a{y)] dy 

- (/(Q(0) - y • 1) - /(Q(0))) • 1 dy + 2n 

< /(Q(0)) . A + /(r • 1) . A - (l - I) (^max^ /(Q(0)) • 



+ f{yl)-ldy + 0{l) 
< /(Q(0)) . A - (l - I) (^^max^ /(Q(0)) • + 2n/(r + 1) + 0(1). 

In above, (a) uses Lipschitz property of Q(-) (as a function of r); (b) follows 
from (1491) and the inequality that for f{x) = loglog(x + e), /(x) + f{y) + 
log 2 > f{x + y) for all x,y £ M+. The 0(1) term is constant, dependent on 
n, and captures the constant from (|49l) . 
Now since A G (1 — e) Conv(X(G)), we obtain 

E,[L(X(t + 1))-L(X(t))] 



< 



-if max /(Q(0))-p') +2n/(T + l) + 0(l) 

2 \p€XiG) J 



< -|/(Qmax(0))+2n/(T + l)+O(l). 
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Therefore, summing r from bi = 6i((5max(0)) to 62 = 62(<3max(0)), we have 
Ex [L{X{b2))-L{X{h))] 

b2-l 

< -|(&2 - 6i)/(g„,ax(0)) + 2n ^ /(r + 1) + 0(62 - 61) 

T=bi 

(51) < -|(62 - 6i)/(Qmax(0)) + 2n(62 - 60/(62) + 0(62 - 61). 

Thus, we obtain 

Ex [L{X(b2))-L{X{0))] 
= Ex [L(X(6i)) - L{X{0))] + Ex [^.(^(62)) - L{X{bi))] 

< Ex [/(Q(6i)) • (Q(6i) - Q(0))] - |(62 - 6i)/(Q^ax(0)) 

62-1 



+2n J2 /(^ + 1) + 0(62 - 61 



T=6i 

(^) e 

< nbi /(Qn,ax(0) + 61)) - -(62 - 6l)/(Qmax(0)) 

(52) +2n(62-6i)/(62) + 0(62-6i), 

where (a) follows from the convexity of L and (b) is due to the 1-Lipschitz 
property of Q. Now if we choose (?(x) = 62 and 

/i(x) = -7i6i /(Q^,,(0)+6i))+|(62-6i)/(Qmax(0))-2n(62-6i)/(62)-O(62-6i), 

the desired inequality follows: 

Ex [L(X(g(x))) - L(X(0))] < -Mx). 

The desired conditions of Lemma [6] can be checked as follows. First observe 
that with respect to (5max(0), the function h scales as 62((5max(0))/((5max(0)) 
due to 62/61 = 6 (log (5max(0)) as per Lemma[9l Further, /i is a function that 
is lower bounded and its value goes to 00 as Qmax{0) goes to 00. Therefore, 
h/g scales as /(Qmax(O)). These propeties will imply the verification condi- 
tions of Lemma [H 



5.3.4. Step Three: Buffered Circuit Switched Network. In this section, we 
prove Lemma [6] for the circuit switched network model. Similar to wireless 
network, we are interested in large enough (3max(0) that satisfies condition 
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of LemmalU Given the state X{t) = (Q(t), z{t)) of the Markov process, we 
shall consider the following Lyapunov function : 



Here R(t) = [Ri{t)] with Ri{t) = Qi{t) + Zi{t) and as before F{x) = 
Jo f{y) (^y- Now we proceed towards finding appropriate functions h and 
g as desired in Lemma [H For any r G Z+, 



L{X{t + 1))-L{X{t)) 

= (F(R(r + l))-F(R(r))).l 
</(R(r + l)).(R(r + l)-R(T)), 

= /(R(r) + A{t, t + 1) - D{t, t + 1)) • (^(r, r + 1) - 7^(t, r + 1)) 
< /(R(r)) • {A{t, r + 1) - I)(t, r + 1)) + \\A{t, r + 1) - D(r, r + 



Given initial state X{0) = x, taking expectation for r, r + 1 G /, we have 



The last equality follows from the fact that arrival process is Poisson with 
rate vector A and R(t) is independent of A{t,t + 1). In addition, the overall 
departure process for any i, Di{-), is governed by a Poisson process of rate 
at most Cinax- Therefore, the second moment of the difference of arrival and 
departure processes in unit time is 0(1). Now, 



(54) [/(R(r)) ■ A] = /(R(0)) • A + [(/(R(t)) - /(R(0))) • A] . 



Ex [/(R(T))-Z)(T,r + l)] 
(55) = E, [/(R(0)) • D{t, t + 1)] + E, [(/(R(r)) - /(R(0))) • D{t, t + 1)] . 



The first term on the right hand side in ()54p can be bounded as 



L{X{t)) = Y,F{R,{t)). 



E^[L{X{T + l))-LiX{T))] 

< Ex [/(R(r)) • Air, r + 1)] - E, [/(R(t)) • Z)(t, r + 1)] 



(53) 



+E^[\\A{t,t + 1)-D{t,t + 1)\\1] 
=Ex [/(R(t)) • A] - E, [/(R(r)) • D{t, t + 1)] + 0(1). 



And 



(56) 
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where the first inequahty is due to A G (1 — e) Com{X) and the second 
inequality fohows from Lemma [7] with the fact that |/j(R(r)) — /j(Q(T))| < 
/(C'max) = 0(1) for all i. On the other hand, the first term in the right hand 
side of (l55l) can be bounded below as 



Ex[/(R(0))-Z)(r,r + 1)] = /(R(0)) • [I?(t, r + 1)] 

r-r+l 



> f{K{0))- £ E^[z{s)] ds 



(57) = ^"^'e^(,)[/(R(0))-^] ds. 

In above, we have used the fact that -Dj(-) is a Poisson process with rate 
given by Zi{-). Further, the second term in the right hand side of (f53|) can be 
bounded as follows. 

E.[||/(R(r))-/(R(0))||i] < E.[/(|R(r))-R(0)|)] + O(l) 

< /(E,[|R(r))-R(0)|]) + O(l) 

< n/(C^axr) + 0(1) 

(58) = 0(/(r)), 

The first inequality follows from f{x + y) < f{x) + f{y) + 2 for any x,y G M_|_. 
This is because log(x + y + e) < log(2; + e) + log(y + e) for any x,y > M+, 
logo + b < 2 + log a + log 6 for any o, 6 > 1 and f{x) = loglog(2; + e). 
The second inequality follows by applying Jensen's inequality for concave 
function /. Combining ([53|) - ([58]) . we obtain 



< 



KMX{t + 1))-L{X{t))] 

(max/(R(0)) • + E,(o) [/(R(0)) • 



£ E^(,) [/(R(0)) • ^] ds + 0{f{T)) 



< 



(a) 
< 



3e 



4 fe/(R(0))-^ 



+ 1^^' (^max/(R(0)) • yj Ms) - 7t{0)\\tv ds + OU{t)) 



-|(^max/(R(0))-2/j +0(/(r)) 



< -|/(Qmax(0))+O(/(T)), 
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where (a) follows from Lemma [H Summing this for t G I = [61 , 62 — 1] j 

(59) 

EMX{b2))-L{X{h))] < -£/(Q^,,(0))(62-6i) + O((62-fei)/(fe2)). 
Therefore, we have 

E^[L{X{b2))-L(X{0))] 
= E^[L{X{bi)) - L{X{0))] + E^[L{X{b2)) - L{X{bi))] 

< E,[/(R(6i)) • (R(6i) - R(0))] + E^[L{X{b2)) - L{X{bi))] 

= ^Ex[/(i?./(6i)) • {Ri{bi) - RiiO))] + E,[L(X(62)) - L{X{h))] 

i 

(b) ^ , , 

< Yl V^AfiR^iblWW^AiRiibl) - RiiOW] + Ex[L(X(62)) - LiXibi))] 

i 

< ^^/(Ex[i2i(&i)])2 + 0(1) -0(61) + Ex[L(X(62)) - L{X{bi))] 

i 

= n/(Q^ax(0) + 0(61)) • 0(61) - |/(Qmax(0))(62 " 5l) 

+ 0((&2-6i)/(62)) 

^ -Mx). 

Here (a) follows from convexity of L; (b) from Cauchy-Schwarz, (c) is due to 
the bounded second moment Ex[|i?i(6i) — Ri{0)\] = 0(6i) as argued earlier 
in the proof and observing that there exists a concave function g such that 

= (7 + 0(1) over IR+, subsequently Jensen's inequality can be applied; (d) 
follows from (]59p . Finally, choose g(x) = 62. 

With these choices of h and g, the desired conditions of Lemma [6] can be 
checked as follows. First observe that with respect to Qmax(O), the function 
h scales as 62(Qmax(0))/(Qmax(0)) due to 62/61 = (log(5max(0)) as per 
Lemma [H Further, /i is a function that is lower bounded and its value goes 
to 00 as Qmax(O) goes to 00. Therefore, h/g scales as /(Qmax(O)). These 
properties will imply the verification conditions of Lemma [6l 

5.3.5. Step Four. For completing the proof of the positive Harris recur- 
rence of both algorithms, it only remains to show that for k > 0, the set 
i?K = {x G X : L(x) < k} is a closed petit. This is because other conditions 
of Lemma [5] follow from Lemma [H And the Step Three exhibited choice of 
Lyapunov function L and desired 'drift' functions h,g. 

To this end, first note that is closed by definition. To establish that 
it is a petit set, we need to find a non-trivial measure fi on {X,13x) and 
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sampling distribution a on Z+ so that for any x G i?^! 

To construct such a measure /i, we shall use the following Lemma. 

Lemma 10 Let the network Markov chain X{-) start with the state x G 
at time i.e. X{0) = x. Then, there exists > 1 and 7k > such that 

J^Prx(X(T) =0) >7k, VxeS«. 

T = l 

Here = (0, 0) G X denote the state where all components of Q are and 
the schedule is the empty independent set. 

Proof. We establish this for wireless network. The proof for circuit switched 
network is identical and we skip the details. Consider any x G B^. Then by 
definition L{x.) < k + 1 for given k > 0. Hence by definition of L(-) it can be 
easily checked that each queue is bounded above by k. Consider some large 
enough (soon to be determined) T^. By the property of Bernoulli (or Poisson 
for circuit switched network) arrival process, there is a positive probability 
^2 > of no arrivals happening to the system during time interval of length 
T^. Assuming that no arrival happens, we will show that in large enough time 
t},, with probability 0}, > each queue receives at least k amount of service; 
and after that in additional time with positive probability 6^ > the 

empty set schedule is reached. This will imply that by defining = t}^ + t^ 
the state G X is reached with probability at least 

7. = elele^ > o. 

And this will immediately imply the desired result of Lemma [TUl To this 
end, we need to show existence of t\,9\ and t^,9'^ with properties stated 
above to complete the proof of Lemma [101 

First, existence of t\, 9\. For this, note that the Markov chain correspond- 
ing to the scheduling algorithm has time varying transition probabilities and 
is irreducible over the space of all independent sets, I{G). If there are no new 
arrivals and initial x G .Bki then clearly queue-sizes are uniformly bounded 
by K. Therefore, the transition probabilities of all feasible transitions for 
this time varying Markov chain is uniformly lower bounded by a strictly 
positive constant (dependent on k, n). It can be easily checked that the 
transition probability induced graph on I{G) has diameter at most 2n and 
Markov chain transits as per Exponential clock of overall rate n. Therefore, 
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it follows that starting from any initial scheduling configuration, there exists 
finite time such that a schedule is reached so that any given queue i is 
scheduled for at least unit amount of time with probability at least 6^ > 0. 
Here, both t^, 9k depend on n, k. Therefore, it follows that in time t\ = Knt^ 
all queues become empty with probability at least 6^ — ■ Next, to 

establish existence of t^, 9^ as desired, observe that once the system reaches 
empty queues, it follows that in the absence of new arrivals the empty sched- 
ule is reached after some finite time with probability ^■^ > by similar 
properties of the Markov chain on I{G) when all queues are 0. Here and 
9^ are dependent on n only. This completes the proof of Lemma (TOj □ 

In what follows. Lemma [TOl will be used to complete the proof that is 
a closed petit. To this end, consider Geometric(l/2) as the sampling distri- 
bution a, i.e. 

a{e) = £>1. 
Let ^0 be the Dirac distribution on element G X. Then, define ^ as 

^ = 2"'^'=7fcdo, that is fj.{-) = 2~'^'^-fkSo{-)- 

Clearly, ^ is non-trivial measure on (X, i3x). With these definitions of a and 
/i. Lemma [TOl immediately implies that for any x E B^,, 

i^a(x,-) >M(-)- 

This establishes that set B^ is a closed petit set. 

6. Discussion. This paper introduced a new randomized scheduling 
algorithm for two constrained queueing network models: wireless network 
and buffered circuit switched network. The algorithm is simple, distributed, 
myopic and throughput optimal. The main reason behind the throughput 
optimality property of the algorithm is two folds: (1) The relation of algo- 
rithm dynamics to the Markovian dynamics over the space of schedules that 
have a certain product-form stationary distribution, and (2) choice of slowly 
increasing weight function loglog(- + e) that allows for an effective time 
scale separation between algorithm dynamics and the queueing dynamics. 
We chose wireless network and buffered circuit switched network model to 
explain the effectiveness of our algorithm because (a) they are becoming of 
great interest \26\ I34j and (b) they represent two different, general class of 
network models: synchronized packet network model and asynchronous flow 
network model. 
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Now we turn to discuss the distributed implementation of our algorithm. 
As described in Section 13. H given the weight information at each wire- 
less node (or ingress of a route), the algorithm completely distributed. The 
weight, as defined in ([3|) (or (j3|)), depends on the local queue-size as well as 
the Q 

max information. As is, Qmax is global information. To keep the expo- 
sition simpler, we have used the precise Qmax information to establish the 
throughput property. However, as remarked earlier in the Section 13.11 (soon 
after ([3])), the Q 

max can be replaced by its appropriate distributed estima- 
tion without altering the throughput optimality property. Such a distributed 
estimation can be obtained through an extremely simple Markovian like al- 
gorithm that require each node to perform broadcast of exactly one number 
in unit time. A detailed description of such an algorithm can be found in 
Section 3.3 of [25] . 

On the other hand, consider the algorithm that does not use Qmax infor- 
mation. That is, instead of ([3]) or dH), let weight be 

w^{t) = f{Qm))- 

We conjecture that this algorithm is throughput optimal. 
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APPENDIX A: A USEFUL LEMIViA 
Lemma 11 Let Pi,P2 G M^^^. Then, 

where M = max{||Pi||oo, ||-Pi||oo}- 

Proof. Using mathematical induction, we first establish that for any fc £ N, 
(^QQ^ II D^ll ^ J^/' Ar 1 1 



\Pl-P2\\oo < k{NMf~'\\Pi- P2\ 



□ 



To this end, the base case k = 1 follows trivially. Suppose it is true for some 
k > 1. Then, the inductive step can be justified as follows. 



I p^+i _ p^+i|| 

Pi ^2 I loo 



Pl(Pl 



Pi 



< 
(a) 

< N \\Pi\ 



Pi (P^ - P| 



[Pi 
+ 



pfc r>k 
^1 ~ ^2 



P2)P^ 



(Pi - P2) P| 
+ N\\Pi-P2\\^ 



P^ 



< NM X kiN M)''~'^\\Pi - P2II00 + iV ||Pi - P2I 



X iV" 



(A; + l)(iVM)'=||Pi-P2 



2 oo- 



In above, (a) follows from an easily verifiable fact that for any Qi,Q2 G 

■mNxN 



igig2iioo<A^ilQiilooiia 



2 loo- 
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We use induction hypothesis to justify (b). Using (fHOl) . we have 



k 

k 



A;! 

k 



Pi — -Pzlloo- 



